Computing systems that use parallel processing divide a single application into multiple processes that can be performed in parallel on a single computing core or can be performed in parallel on multiple different computing cores. In order to function cooperatively, the parallel processes often need to communicate with each other to pass data and to synchronize their operation. In many parallel computing systems, this communication is achieved using a Message Passing Interface (MPI) library.
MPI is a specification for a library of functions. There are several different implementations of MPI libraries but each implementation adheres to at least one version of the MPI specification. The MPI specifications use objects called communicators and groups to define collections of processes that may communicate with each other. Most MPI functions require a communicator to be specified as an argument and when a communicator is specified for a function, the function is limited to interacting with processes that are part of the communicator and cannot interact with processes outside of the communicator. Each process that is assigned to a communicator is given an identifier referred to as the rank of the process in the communicator, and this rank is required by functions that perform point-to-point communication.
Many of the MPI functions can be divided into blocking and non-blocking functions where a process will stop executing and wait for an event after calling a blocking function while the process will not stop executing after calling a non-blocking function.
Most MPI libraries include at least the following functions: MPI_INIT, MPI_SEND, MPI_RECV, MPI_BARRIER, MPI_BCAST, MPI_REDUCE, and MPI_FINALIZE. MPI_INIT initializes the MPI execution environment. This function must be called in every MPI program, must be called before any other MPI functions and must be called only once in an MPI program. MPI_SEND is a basic blocking send function that sends a message to another process. The function returns only after the buffer containing the message to be sent is free for reuse. MPI_SEND requires the designation of a communicator and the rank of the process that is to receive the message. MPI_RECV is a blocking receive function that will block the process until the requested data is available in a buffer set in the MPI_RECV call. MPI_BARRIER is a synchronization operation that designates a group of processes that must all reach a same synchronization point before any of them can continue. Each process that reaches a MPI_Barrier call blocks until all processes in the group reach the same MPI_Barrier call. Then all of the processes are free to proceed. MPI_REDUCE is a collective computation operation that applies a reduction operation on all processes in a group of processes and places the result in one process known as the root process. MPI_FINALIZE terminates the MPI execution environment. This function should be the last MPI routine called in every process that uses an MPI library—no other MPI routines may be called after it.
The discussion above is merely provided for general background information and is not intended to be used as an aid in determining the scope of the claimed subject matter. The claimed subject matter is not limited to implementations that solve any or all disadvantages noted in the background.